Networked servers can provide services to other networked hosts. Some of these networked hosts attempt to exploit a server by taking advantage of security loopholes. One exploitation method uses a security hole to inject malicious code into the server memory, and executes the malicious code to exploit the server. This malicious code can search the data structures of an application, a library, or an operating system component to find and utilize server system resources for the purpose of interrupting these services (such as denial of service attacks), to access sensitive information residing on the server, or to perform other malicious steps.
FIG. 1 illustrates one configuration of a malicious host 110 requesting services from a trusted host 120 providing network services. The trusted host 120 is a computer server receiving service requests 112 over a network and providing services 114 such as web, mail, and file sharing. Malicious code is injected from the malicious host 110 into the trusted host 120 using a service request. The malicious host 110 requests a service 112 in which the request encapsulates malicious code. A maliciously structured service request can exploit buffer overflow techniques resulting in the malicious code 140 being loaded into memory 150 of the trusted host 120. Traditional methods of detecting malicious software include special purpose hardware 130 such as a firewall that inspects packets within a network data stream. The packet inspection tools include processing protocols such as HTTP, in which packet payloads are for the most part plain text data. Except for image data or graphics, which can be detected and identified, a firewall will inspect a packet for unexpected binary data. When packet inspection detects unexpected binary data, an indicator of potentially executable and malicious code, the firewall 130 can isolate either the host making the service request or the stream of transmitted data. The disadvantage of this solution is that additional hardware is required. Further, blindly inspecting all packets is processing intensive. Additionally, deep packet inspection techniques require the assembly of payload data streams spread across multiple network packets and result in increased data latency.
Other malicious software identification techniques check for patterns in files and data streams against known virus signatures. The limitation of this technique is that these signatures need to be kept current and are inherently out of date when a new virus is developed.